Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

clear_payloads func #551

Merged
merged 8 commits into from
Apr 9, 2024
Merged

clear_payloads func #551

merged 8 commits into from
Apr 9, 2024

Conversation

volovyks
Copy link
Collaborator

@volovyks volovyks commented Apr 1, 2024

One of our envs is stuck because it has >8 payloads. PPPro because it had 8 payloads in the past. I've added a clean function that should be called when this new version of the contract is deployed. @ppca please, do that for all our envs. And for v5.multichain-mpc-dev.testnet specifically.

@ChaoticTempest I've tried to add a read/write lock to requests, but borsh was not happy about that. Not sure if that is necessary at all.

Copy link

github-actions bot commented Apr 1, 2024

Terraform Feature Environment (dev-551)

Terraform Initialization ⚙️success

Terraform Apply success

Show Apply Plan

data.external.git_checkout: Reading...
data.external.git_checkout: Read complete after 0s [id=-]
data.google_compute_network.dev_network: Reading...
data.google_compute_subnetwork.dev_subnetwork: Reading...
google_service_account.service_account: Refreshing state... [id=projects/pagoda-discovery-platform-dev/serviceAccounts/mpc-recovery-dev-551@pagoda-discovery-platform-dev.iam.gserviceaccount.com]
data.google_compute_subnetwork.prod_subnetwork: Reading...
data.google_compute_network.prod_network: Reading...
google_service_account_iam_binding.serivce-account-iam: Refreshing state... [id=projects/pagoda-discovery-platform-dev/serviceAccounts/mpc-recovery-dev-551@pagoda-discovery-platform-dev.iam.gserviceaccount.com/roles/iam.serviceAccountUser]
google_secret_manager_secret_iam_member.cipher_key_secret_access[1]: Refreshing state... [id=projects/pagoda-discovery-platform-dev/secrets/mpc-cipher-1-dev/roles/secretmanager.secretAccessor/serviceAccount:mpc-recovery-dev-551@pagoda-discovery-platform-dev.iam.gserviceaccount.com]
google_secret_manager_secret_iam_member.fast_auth_partners_secret_access: Refreshing state... [id=projects/pagoda-discovery-platform-dev/secrets/mpc-fast-auth-partners-dev/roles/secretmanager.secretAccessor/serviceAccount:mpc-recovery-dev-551@pagoda-discovery-platform-dev.iam.gserviceaccount.com]
google_project_iam_member.service-account-datastore-user: Refreshing state... [id=pagoda-discovery-platform-dev/roles/datastore.user/serviceAccount:mpc-recovery-dev-551@pagoda-discovery-platform-dev.iam.gserviceaccount.com]
google_secret_manager_secret_iam_member.cipher_key_secret_access[2]: Refreshing state... [id=projects/pagoda-discovery-platform-dev/secrets/mpc-cipher-2-dev/roles/secretmanager.secretAccessor/serviceAccount:mpc-recovery-dev-551@pagoda-discovery-platform-dev.iam.gserviceaccount.com]
google_secret_manager_secret_iam_member.cipher_key_secret_access[0]: Refreshing state... [id=projects/pagoda-discovery-platform-dev/secrets/mpc-cipher-0-dev/roles/secretmanager.secretAccessor/serviceAccount:mpc-recovery-dev-551@pagoda-discovery-platform-dev.iam.gserviceaccount.com]
data.google_compute_subnetwork.prod_subnetwork: Read complete after 0s [id=projects/pagoda-shared-infrastructure/regions/us-east1/subnetworks/cloudrun-main-prod-us-east1]
google_secret_manager_secret_iam_member.account_creator_secret_access: Refreshing state... [id=projects/pagoda-discovery-platform-dev/secrets/mpc-recovery-account-creator-sk-dev/roles/secretmanager.secretAccessor/serviceAccount:mpc-recovery-dev-551@pagoda-discovery-platform-dev.iam.gserviceaccount.com]
data.google_compute_subnetwork.dev_subnetwork: Read complete after 0s [id=projects/pagoda-shared-infrastructure/regions/us-east1/subnetworks/cloudrun-main-dev-us-east1]
google_secret_manager_secret_iam_member.secret_share_secret_access[0]: Refreshing state... [id=projects/pagoda-discovery-platform-dev/secrets/mpc-sk-share-0-dev/roles/secretmanager.secretAccessor/serviceAccount:mpc-recovery-dev-551@pagoda-discovery-platform-dev.iam.gserviceaccount.com]
data.google_compute_network.dev_network: Read complete after 0s [id=projects/pagoda-shared-infrastructure/global/networks/dev]
google_secret_manager_secret_iam_member.secret_share_secret_access[1]: Refreshing state... [id=projects/pagoda-discovery-platform-dev/secrets/mpc-sk-share-1-dev/roles/secretmanager.secretAccessor/serviceAccount:mpc-recovery-dev-551@pagoda-discovery-platform-dev.iam.gserviceaccount.com]
data.google_compute_network.prod_network: Read complete after 0s [id=projects/pagoda-shared-infrastructure/global/networks/prod]
google_secret_manager_secret_iam_member.secret_share_secret_access[2]: Refreshing state... [id=projects/pagoda-discovery-platform-dev/secrets/mpc-sk-share-2-dev/roles/secretmanager.secretAccessor/serviceAccount:mpc-recovery-dev-551@pagoda-discovery-platform-dev.iam.gserviceaccount.com]
module.signer[1].google_cloud_run_v2_service.signer: Refreshing state... [id=projects/pagoda-discovery-platform-dev/locations/us-east1/services/mpc-recovery-signer-1-dev-551]
module.signer[2].google_cloud_run_v2_service.signer: Refreshing state... [id=projects/pagoda-discovery-platform-dev/locations/us-east1/services/mpc-recovery-signer-2-dev-551]
module.signer[0].google_cloud_run_v2_service.signer: Refreshing state... [id=projects/pagoda-discovery-platform-dev/locations/us-east1/services/mpc-recovery-signer-0-dev-551]
module.signer[0].google_cloud_run_v2_service_iam_member.allow_all: Refreshing state... [id=projects/pagoda-discovery-platform-dev/locations/us-east1/services/mpc-recovery-signer-0-dev-551/roles/run.invoker/allUsers]
module.signer[1].google_cloud_run_v2_service_iam_member.allow_all: Refreshing state... [id=projects/pagoda-discovery-platform-dev/locations/us-east1/services/mpc-recovery-signer-1-dev-551/roles/run.invoker/allUsers]
module.signer[2].google_cloud_run_v2_service_iam_member.allow_all: Refreshing state... [id=projects/pagoda-discovery-platform-dev/locations/us-east1/services/mpc-recovery-signer-2-dev-551/roles/run.invoker/allUsers]
module.leader.google_cloud_run_v2_service.leader: Refreshing state... [id=projects/pagoda-discovery-platform-dev/locations/us-east1/services/mpc-recovery-leader-dev-551]
module.leader.google_cloud_run_v2_service_iam_member.allow_all: Refreshing state... [id=projects/pagoda-discovery-platform-dev/locations/us-east1/services/mpc-recovery-leader-dev-551/roles/run.invoker/allUsers]

Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
  ~ update in-place

Terraform will perform the following actions:

  # module.leader.google_cloud_run_v2_service.leader will be updated in-place
  ~ resource "google_cloud_run_v2_service" "leader" {
        id                      = "projects/pagoda-discovery-platform-dev/locations/us-east1/services/mpc-recovery-leader-dev-551"
        name                    = "mpc-recovery-leader-dev-551"
        # (17 unchanged attributes hidden)

      ~ template {
            # (6 unchanged attributes hidden)

          ~ containers {
              ~ image   = "us-east1-docker.pkg.dev/pagoda-discovery-platform-dev/mpc-recovery/mpc-recovery-dev:f8cca7031217cab32add3735bd4ec8f65f7a6acb" -> "us-east1-docker.pkg.dev/pagoda-discovery-platform-dev/mpc-recovery/mpc-recovery-dev:bc9e764856b7fbfb4c10aec3a99e08644b486361"
                # (2 unchanged attributes hidden)

                # (16 unchanged blocks hidden)
            }

            # (2 unchanged blocks hidden)
        }

        # (1 unchanged block hidden)
    }

  # module.signer[0].google_cloud_run_v2_service.signer will be updated in-place
  ~ resource "google_cloud_run_v2_service" "signer" {
        id                      = "projects/pagoda-discovery-platform-dev/locations/us-east1/services/mpc-recovery-signer-0-dev-551"
        name                    = "mpc-recovery-signer-0-dev-551"
        # (17 unchanged attributes hidden)

      ~ template {
            # (6 unchanged attributes hidden)

          ~ containers {
              ~ image   = "us-east1-docker.pkg.dev/pagoda-discovery-platform-dev/mpc-recovery/mpc-recovery-dev:f8cca7031217cab32add3735bd4ec8f65f7a6acb" -> "us-east1-docker.pkg.dev/pagoda-discovery-platform-dev/mpc-recovery/mpc-recovery-dev:bc9e764856b7fbfb4c10aec3a99e08644b486361"
                # (2 unchanged attributes hidden)

                # (11 unchanged blocks hidden)
            }

            # (2 unchanged blocks hidden)
        }

        # (1 unchanged block hidden)
    }

  # module.signer[1].google_cloud_run_v2_service.signer will be updated in-place
  ~ resource "google_cloud_run_v2_service" "signer" {
        id                      = "projects/pagoda-discovery-platform-dev/locations/us-east1/services/mpc-recovery-signer-1-dev-551"
        name                    = "mpc-recovery-signer-1-dev-551"
        # (17 unchanged attributes hidden)

      ~ template {
            # (6 unchanged attributes hidden)

          ~ containers {
              ~ image   = "us-east1-docker.pkg.dev/pagoda-discovery-platform-dev/mpc-recovery/mpc-recovery-dev:f8cca7031217cab32add3735bd4ec8f65f7a6acb" -> "us-east1-docker.pkg.dev/pagoda-discovery-platform-dev/mpc-recovery/mpc-recovery-dev:bc9e764856b7fbfb4c10aec3a99e08644b486361"
                # (2 unchanged attributes hidden)

                # (11 unchanged blocks hidden)
            }

            # (2 unchanged blocks hidden)
        }

        # (1 unchanged block hidden)
    }

  # module.signer[2].google_cloud_run_v2_service.signer will be updated in-place
  ~ resource "google_cloud_run_v2_service" "signer" {
        id                      = "projects/pagoda-discovery-platform-dev/locations/us-east1/services/mpc-recovery-signer-2-dev-551"
        name                    = "mpc-recovery-signer-2-dev-551"
        # (17 unchanged attributes hidden)

      ~ template {
            # (6 unchanged attributes hidden)

          ~ containers {
              ~ image   = "us-east1-docker.pkg.dev/pagoda-discovery-platform-dev/mpc-recovery/mpc-recovery-dev:f8cca7031217cab32add3735bd4ec8f65f7a6acb" -> "us-east1-docker.pkg.dev/pagoda-discovery-platform-dev/mpc-recovery/mpc-recovery-dev:bc9e764856b7fbfb4c10aec3a99e08644b486361"
                # (2 unchanged attributes hidden)

                # (11 unchanged blocks hidden)
            }

            # (2 unchanged blocks hidden)
        }

        # (1 unchanged block hidden)
    }

Plan: 0 to add, 4 to change, 0 to destroy.
module.signer[2].google_cloud_run_v2_service.signer: Modifying... [id=projects/pagoda-discovery-platform-dev/locations/us-east1/services/mpc-recovery-signer-2-dev-551]
module.signer[1].google_cloud_run_v2_service.signer: Modifying... [id=projects/pagoda-discovery-platform-dev/locations/us-east1/services/mpc-recovery-signer-1-dev-551]
module.signer[0].google_cloud_run_v2_service.signer: Modifying... [id=projects/pagoda-discovery-platform-dev/locations/us-east1/services/mpc-recovery-signer-0-dev-551]
module.signer[0].google_cloud_run_v2_service.signer: Still modifying... [id=projects/pagoda-discovery-platform-dev/...services/mpc-recovery-signer-0-dev-551, 10s elapsed]
module.signer[1].google_cloud_run_v2_service.signer: Still modifying... [id=projects/pagoda-discovery-platform-dev/...services/mpc-recovery-signer-1-dev-551, 10s elapsed]
module.signer[2].google_cloud_run_v2_service.signer: Still modifying... [id=projects/pagoda-discovery-platform-dev/...services/mpc-recovery-signer-2-dev-551, 10s elapsed]
module.signer[2].google_cloud_run_v2_service.signer: Still modifying... [id=projects/pagoda-discovery-platform-dev/...services/mpc-recovery-signer-2-dev-551, 20s elapsed]
module.signer[1].google_cloud_run_v2_service.signer: Still modifying... [id=projects/pagoda-discovery-platform-dev/...services/mpc-recovery-signer-1-dev-551, 20s elapsed]
module.signer[0].google_cloud_run_v2_service.signer: Still modifying... [id=projects/pagoda-discovery-platform-dev/...services/mpc-recovery-signer-0-dev-551, 20s elapsed]
module.signer[0].google_cloud_run_v2_service.signer: Modifications complete after 20s [id=projects/pagoda-discovery-platform-dev/locations/us-east1/services/mpc-recovery-signer-0-dev-551]
module.signer[2].google_cloud_run_v2_service.signer: Modifications complete after 20s [id=projects/pagoda-discovery-platform-dev/locations/us-east1/services/mpc-recovery-signer-2-dev-551]
module.signer[1].google_cloud_run_v2_service.signer: Modifications complete after 21s [id=projects/pagoda-discovery-platform-dev/locations/us-east1/services/mpc-recovery-signer-1-dev-551]
module.leader.google_cloud_run_v2_service.leader: Modifying... [id=projects/pagoda-discovery-platform-dev/locations/us-east1/services/mpc-recovery-leader-dev-551]
module.leader.google_cloud_run_v2_service.leader: Still modifying... [id=projects/pagoda-discovery-platform-dev/...1/services/mpc-recovery-leader-dev-551, 10s elapsed]
module.leader.google_cloud_run_v2_service.leader: Still modifying... [id=projects/pagoda-discovery-platform-dev/...1/services/mpc-recovery-leader-dev-551, 20s elapsed]
module.leader.google_cloud_run_v2_service.leader: Modifications complete after 20s [id=projects/pagoda-discovery-platform-dev/locations/us-east1/services/mpc-recovery-leader-dev-551]

Apply complete! Resources: 0 added, 4 changed, 0 destroyed.

Outputs:

leader_node = "https://mpc-recovery-leader-dev-551-7tk2cmmtcq-ue.a.run.app"

Pusher: @volovyks, Action: pull_request, Working Directory: ``, Workflow: Terraform Feature Env

URL: https://mpc-recovery-leader-dev-551-7tk2cmmtcq-ue.a.run.app

@volovyks volovyks requested review from ChaoticTempest and ppca April 1, 2024 16:24
contract/src/lib.rs Outdated Show resolved Hide resolved
@ChaoticTempest
Copy link
Member

I've tried to add a read/write lock to requests, but borsh was not happy about that. Not sure if that is necessary at all.

How were you using locks? Locks in contract code should not be required at all. Also, you can't serialize a lock unless it's a filelock but that shouldn't be used here in contract code either

@volovyks volovyks requested a review from ChaoticTempest April 2, 2024 09:41
@volovyks
Copy link
Collaborator Author

volovyks commented Apr 2, 2024

@ChaoticTempest yep, just looking for a reason why the counter can be off. Looks like resetting should help.

}

#[private]
pub fn clean_payloads(&mut self, payloads: Vec<[u8; 32]>, counter: u32) {
Copy link
Contributor

@ppca ppca Apr 2, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why specify the payloads instead of clearing all the requests in the pending_requests? is it because of LookupMap?

Copy link
Contributor

@ppca ppca Apr 2, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've used

#[private]
    pub fn clean_payloads(&mut self) {
        log!("clean_payloads");
        self.pending_requests = LookupMap::new(b"m");
        self.request_counter = 0;
    }

to clean the payloads on dev for now.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, that is fine. But the old payloads are still there.
LookupMap can not clean its key-value pairs. It is optimized for storage and access speed.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wait, why would the old payloads still be there? I thought self.pending_requests = LookupMap::new(b"m"); self.request_counter = 0; will reset it?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is how insert and remove works in LookupMap

    /// Inserts a serialized key-value pair into the map.
    /// If the map did not have this key present, `None` is returned. Otherwise returns
    /// a serialized value. Note, the keys that have the same hash value are undistinguished by
    /// the implementation.
    pub fn insert_raw(&mut self, key_raw: &[u8], value_raw: &[u8]) -> Option<Vec<u8>> {
        let storage_key = self.raw_key_to_storage_key(key_raw);
        if env::storage_write(&storage_key, value_raw) {
            Some(env::storage_get_evicted().unwrap())
        } else {
            None
        }
    }

    /// Removes a serialized key from the map, returning the serialized value at the key if the key
    /// was previously in the map.
    pub fn remove_raw(&mut self, key_raw: &[u8]) -> Option<Vec<u8>> {
        let storage_key = self.raw_key_to_storage_key(key_raw);
        if env::storage_remove(&storage_key) {
            Some(env::storage_get_evicted().unwrap())
        } else {
            None
        }
    }

LookupMap is not aware of its key-value pairs. They are stored directly on the contract memory which is also a map. The key for storing values is "LopkupMap prefix + provided key". It is not possible to clear the Lookup map without passing the keys.

@volovyks volovyks requested a review from ppca April 3, 2024 11:26
@volovyks
Copy link
Collaborator Author

volovyks commented Apr 3, 2024

@ChaoticTempest @ppca ready for review

Copy link
Member

@ChaoticTempest ChaoticTempest left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

use primitives::ParticipantInfo;
use primitives::{CandidateInfo, Candidates, Participants, PkVotes, Votes};
use std::collections::{BTreeMap, HashSet};

const GAS_FOR_SIGN_CALL: Gas = Gas::from_gas(3 * 100_000_000_000_000);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could also just do Gas::from_tgas(300)

@volovyks volovyks merged commit c6e1d66 into develop Apr 9, 2024
6 checks passed
@volovyks volovyks deleted the serhii/contract-request-count-fix branch April 9, 2024 10:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants